Skip to content

Epic: ORCID-first CLI sign-in and service accounts (#1272) - #1291

Merged
neuromechanist merged 10 commits into
devfrom
feature/issue-1272-epic-orcid-cli-signin
Sep 8, 2026
Merged

neuromechanist merged 10 commits into
devfrom
feature/issue-1272-epic-orcid-cli-signin

Conversation

@neuromechanist

@neuromechanist neuromechanist commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1272.

Epic branch feature/issue-1272-epic-orcid-cli-signin into dev, regular merge to keep the per-phase history. Four code phases, each squash-merged into the epic branch after a five-lens review round and a fix round; the docs phase lands as a separate docs PR after the release.

Phases

  • Phase 1 (Phase 1: Backend device authorization flow #1281, PR Phase 1: Backend device authorization flow #1287): the device authorization grant on the backend (RFC 8628): POST /auth/device/{start,token,lookup,confirm,deny}, migration 0081 device_codes with the device code stored hashed, SQL-side timestamps, a 120 s confirm grace, the key minted only when the CLI collects it, and named per-machine keys via GET/POST /auth/keys and DELETE /auth/keys/:id (ADR 0047).
  • Phase 2 (Phase 2: Website authorize page and CLI keys card #1282, website PR CLI authorize page and Settings CLI keys card website#317, already on staging): the /cli/authorize page on the app host, which pins the request origin and uses post-redirect-get, the Settings "CLI keys" card, and next carried through ORCID completion for a brand-new account (website ADR 0016).
  • Phase 3 (Phase 3: CLI login and signup on the device flow #1283, PR Phase 3: CLI login and signup on the device flow #1289): nemar auth login on the device flow: browser opened as a convenience, the printed link and code as the mechanism, --no-open and NEMAR_NO_BROWSER for headless hosts, a poll that survives Ctrl-C cleanly; --key as the paste path; nemar auth signup as login plus profile completion driven by profile_gaps; nemar auth keys list|create|revoke; logout revoking only this machine's device-minted key by default; config file 0600 with tagged key fields; retrieve-key and regenerate-key deprecated for one release; device_code redacted from --debug bundles.
  • Phase 4 (Phase 4: Account kinds: person, service, test #1284, PR Phase 4: Account kinds: person, service, test #1290): account kinds person, service, test (migration 0082); the ORCID gap exempts non-person kinds across all five profile-gap readers; the device flow refuses non-person accounts with service_account; owner-only POST/GET/DELETE /admin/users/:username/keys and POST /admin/users/:username/kind with typed refusals; nemar admin kind, nemar admin keys, nemar admin users --kind, and a nemar admin doctor check for the migrated operational accounts; test personas may create only xx datasets on production (ADR 0048).
  • Phase 5 (Phase 5: Docs and password-login deprecation notice #1285, docs companion nemarOrg/docs#20): docs.nemar.org rewritten around nemar auth login, the headless case, keys per machine, the account-kinds admin page, and the deprecation notice; open as draft PR nemarOrg/docs#23, merged after the release.

Migrations 0081 and 0082

Additive: one new table and one defaulted column with a CHECK constraint. 0082's data half is keyed by username COLLATE NOCASE with a deleted_at IS NULL gate: nemarOwner, nemarAdmin, test-admin, test-owner become service; cool-vibers and the four test-* fixtures become test; test-web stays person. The production spellings were verified read-only against the live catalog before phase 4 merged, and a kind change never revokes keys, so a wrong row is corrected with nemar admin kind rather than a migration.

Deploy and release order

  1. This merge deploys the dev worker and applies 0081 and 0082 to nemar-db-dev.
  2. The live-tier tests, which probe-skip until the routes exist on the dev worker, run for real on this push. Then the staging walk-throughs for phases 1 to 4 on test.nemar.org against api-test.nemar.org with a real ORCID account, including a headless nemar auth login --no-open over SSH.
  3. Promote website staging to main first, so the authorize page is live before any released CLI prints its link; then release dev to main here.
  4. Mark nemarOrg/docs#23 ready and merge it once the release is out.
  5. Post-deploy against production: nemar admin doctor confirms the four operational accounts carry service; mint owner keys with nemar admin keys create where a service account needs one.

Follow-ups filed outside this epic

#1286 (ISO versus SQL timestamp comparison found during phase 1), website #318 (the kind rule transcribed into the website's gap copy). Docs #21 and #22 (git-derived page dates) already merged.

* feat: device-auth contract vocabulary

* feat: device_codes table and service SQL

* refactor: hoist appBase and clientIp

* feat: device authorization routes

* feat: named API key routes

* docs: ADR 0047 and AGENTS.md device flow note

* test: live device flow against the dev worker

* feat: typed token error union and refusal schema

* fix: gate paste-key mint like the device mint

* fix: safe mint read-backs and revoke pre-read

* docs: fix rate-limit arithmetic, ADR cites

* docs: semantic line breaks in ADR 0047 note
* feat: device-flow client error plumbing

prefersMessage learns the device grant/refusal code vocabularies so
message wins; ApiError gains optional code/reason set post-construction;
debug-log redacts device_code. Part of #1283 (epic #1272 phase 3).

* feat: add device auth and named-key API functions

startDeviceAuth, pollDeviceToken, listApiKeys, createApiKey,
revokeApiKey, suggestUsername; LoginResponse.user.username is now
nullable (ADR 0047). Part of #1283 (epic #1272 phase 3).

* feat: secure config file and add account-keying helpers

configFileMode 0600 plus a one-time chmod migration for an existing
file; accountKeyFor, findAccountKeyByEmail, upsertAccount, and the
keyId/keyName/keyCreatedAt/keySource schema fields. Part of #1283
(epic #1272 phase 3).

* feat: guard confirm prompts against non-interactive stdin

confirm()/confirmWithInput() check process.stdin.isTTY before ever
calling inquirer: under stdin: "ignore" inquirer never settles and the
process dies uncatchably at exit. Part of #1283 (epic #1272 phase 3).

* feat: device-flow login (nemar auth login)

New src/lib/device-login.ts drives the device authorization grant:
print the URL and code, try the browser, poll for the key with a
SIGINT-cancellable, timeout-composed poll loop. loginAction rewired
onto it (the --key/NEMAR_API_KEY path stays as validated paste-key
fallback), sharing writeSignedInAccount/applyServerUser with
statusAction --refresh (now renames on refresh and prints the Key:
line) and the profile self-service commands. Exit hook suppresses the
bug-report nudge for 130 (Ctrl-C during the poll). storeAccount()
removed, its one caller moved. Part of #1283 (epic #1272 phase 3).

* feat: revoke this machine's key on logout

Default: DELETE /auth/keys/current only for a device-sourced key; a
pasted or password-era key is kept, with where to revoke it deliberately.
--revoke-key forces, --no-revoke-key skips; --all revokes each stored
account with its own bearer. Part of #1283 (epic #1272 phase 3).

* feat: add nemar auth keys group

list (default), create <name>, revoke <id|current> over GET/POST
/auth/keys and DELETE /auth/keys/:id|current. Part of #1283 (epic
#1272 phase 3).

* feat: backend touches for CLI signup completion

GET /auth/profile/username-suggestion accepts a bearer through
resolveActingAccount, reading the name by actor.id; the ORCID
callback's brand-new-account redirect to /auth/orcid/complete carries
?next= when it names somewhere other than '/'. Part of #1283 (epic
#1272 phase 3).

* feat: signup completion driven by profile_gaps

nemar auth signup runs the device flow, then fills in username,
github_username, city, country via flag or prompt in one PATCH
/auth/profile, requests upload access unless --no-upload-access, and
reports anything the CLI cannot set. Non-TTY stdin names the missing
flags and exits 1 before any prompt. collectResearcherName and
lookupOrcidName are gone with the ORCID/password form. Part of #1283
(epic #1272 phase 3).

* feat: deprecate retrieve-key and regenerate-key

Both print a yellow sentence pointing at 'nemar auth login' before
their first prompt; regenerate-key also names that it revokes the key
on every machine, unlike logout's machine-scoped default. Part of
#1283 (epic #1272 phase 3).

* test: local-tier device-flow CLI coverage

test/auth-device-cli.test.ts (41 cases: URL/code/browser, polling
cadence, terminal answers, success writes the account, stale-probe
re-login, active-account re-login, SIGINT, parallel logins, the
server going away and back, --key, status file mode and rename,
logout, keys, signup completion, deprecations, root shortcuts,
--debug redaction) and test/device-login.unit.test.ts (machineName,
pollDeadlineMs, accountKeyFor). Fixes a bug the stale-probe test
caught: writeSignedInAccount was skipping the old device key's
revoke entirely when the preflight probe found it dead, instead of
only skipping the confirmation line. Part of #1283 (epic #1272
phase 3).

* test: live-tier device-flow CLI coverage

test/auth-device-cli-live.test.ts drives the real CLI subprocess
against a deployed backend (TEST_API_URL), streaming stdout to catch
the printed code and acting on the browser side through seedWebUser
plus the passwordless dev_code sign-in: the full login/status/keys/
logout loop, deny, a pending account's confirm refusal, a mid-flight
revoke, and signup with flags. Prod-guarded and deploy-probe-skipped
like test/auth-device-flow.test.ts; the from-scratch ORCID signup
path and the service-account case (test.todo) are out of reach for
an automated test and are recorded for the manual walk-through on
#1283. Part of #1283 (epic #1272 phase 3).

* docs: CLI device sign-in

README quick start, auth command list, environment variables
(NEMAR_NO_BROWSER; drops the fictional NEMAR_API_URL), and
troubleshooting; AGENTS.md's user flow and CLI commands table; ADR
0047 Consequences for decisions 5-7, 10, 12 (account keying, the
config file's 0600 mode, same-machine key replacement, logout's
scoped revoke, and the retrieve-key/regenerate-key deprecation
replacing the ADR's deferred call). Part of #1283 (epic #1272
phase 3).

* fix: keep Welcome back on the paste-key path

writeSignedInAccount's greeting now depends on source: --key/
NEMAR_API_KEY keeps the pre-phase-3 "Welcome back, <name>!" (a person
pasting a key already holds an account); the device flow keeps the
new "Welcome, <name>!". Pinned in test/auth-device-cli.test.ts: the
--key good case asserts "Welcome back", a device success asserts
"Welcome," and not "Welcome back". Fixes the integration-dev
failure on PR #1289 (test/cli.test.ts:230).

* fix: cache sandbox dataset id on login

applyServerUser now caches sandbox_dataset_id when the server sends
one, restoring behavior the paste path had before this phase.
Pinned through both login paths with a returning trained account;
the stand-in's defaultUser() no longer defaults to a fresh account,
so a caller that forgets to cache it fails loudly.

* fix: classify poll errors and stop after repeats

Only a network failure (statusCode 0) prints the "unreachable"
retry line and waits out the deadline. Any other thrown error --
a 5xx, a 200 that fails deviceTokenSuccessSchema, or a 400 with an
unrecognized code -- is now counted; three in a row end the poll
with a new "server_error" outcome instead of retrying silently
until the deadline. A success resets the count.

Also spreads the success payload before the "kind" discriminant
({ ...result.data, kind: "success" }) so a passthrough field can
never overwrite it.

* feat: parse token errors against the contract

pollDeviceToken now re-parses the token endpoint's 400 body against
deviceTokenErrorSchema (a discriminated union) instead of probing
error/reason as loose strings. reason is typed
DeviceAuthRefusalCode | undefined and is never backfilled from
error -- it is undefined only when the body itself failed to
validate.

A 400 whose error is still a terminal grant code this build
recognizes ends the poll even when the body fails validation,
using the body's own message when present, else
DEVICE_GRANT_MESSAGES[error].

ApiError gains rawBody (the whole parsed error body), set by
request() alongside the existing code/reason fields, so a caller
that needs the full contract shape has it without every other
caller paying for a second body it never asked for.

* fix: tagged key fields and honest account keys

Config's key fields (keyId/keyName/keyCreatedAt/keySource) become a
discriminated union (KeyFields in lib/config.ts): a "device" entry
always carries all three, a "paste" entry carries none, and absent
keySource is a password-era account. writeSignedInAccount takes a
single tagged keyInfo parameter instead of two separately-typed
ones that happened to always agree, so the invalid "device with no
key" state cannot be constructed. A paste login explicitly clears a
stale device key's fields rather than merely omitting them, so a
re-login on a different path from the same machine cannot leave a
no-longer-current keyId behind.

AccountInfo gains "key" (the accounts-map key) and makes "username"
honestly optional -- an email-keyed entry has none. switchAction's
picker, logoutAction --all, and the "Other accounts" status line
now use "key" to look up and revoke accounts instead of a username
that could be undefined; --all previously skipped an email-keyed
account entirely (switchAccount(undefined) finds nothing), which a
new regression test with a mixed username/email-keyed fleet covers.

Also prints a visible warning when the same-machine key revoke on
login fails, instead of swallowing the error silently.

* fix: visible failures on revoke and config write

findAccountKeyByEmail returns undefined (not a guess) when two
stored entries share an email, and dlogs the ambiguity -- a state
this build never writes on its own, so guessing which one a
re-login means risks merging the wrong account's key and cached
fields into it. upsertAccount's caller lands the login as a fresh
entry instead, same as no match.

writeSignedInAccount now catches upsertAccount throwing (a corrupt
store, an unwritable config dir): the sign-in already succeeded
server-side, so a just-minted device key is best-effort revoked
with itself as bearer (new revokeApiKeyWithBearer, since nothing
was ever saved to authenticate the ordinary way) before reporting
the failure and exiting 1, instead of an uncaught exception.

* docs: drop plan citations, fix help and ADR text

Removes every "(decision N)" citation from the phase 3 device-flow
files: each either cites ADR 0047 by content where the ADR actually
covers the rule, or states the rule plainly where it does not (the
plan's own numbering is not something a future reader can verify
against). Fixes the matching phase 1 leftover in
backend/src/services/device-auth.ts.

AGENTS.md's CLI table now shows -k/--key on login only, not signup.
The auth/keys group verbose help spells out that a browser-based
login mints a key while --key/NEMAR_API_KEY only stores one.
retrieve-key/regenerate-key's verbose help gains the deprecation
sentence and regenerate-key's every-machine warning.
guardNonInteractive's docstring now points at confirm()'s guard
instead of independently restating the inquirer failure mode.
test/auth-device-cli-live.test.ts cites
.github/workflows/deploy-backend.yml's push trigger instead of
"AGENTS.md release pipeline" for why the dev worker deploys only
from dev.

config.ts: a named CONFIG_FILE_MODE constant replaces two 0o600
literals; "conf's pre-13 default" becomes "conf's default file mode
(0o666, unrelated to the conf version)" in both config.ts and ADR
0047. ADR 0047 also names MAX_LIVE_API_KEYS where the 25-key cap is
discussed, rejoins a mid-clause line break at the em dash, and
splits an over-long line before its parenthetical.

* test: review coverage for device login

Live test: seedWebUser now returns the seeded row's id, and the
mid-flight-revoke test uses it against POST /admin/revoke/by-id/:id
with an asserted 200 instead of a best-effort, swallowed call.

CLI test: a password-era flat config (no "accounts" key) is migrated
in place, read by auth status (with its password-era Key: line),
and merged into by a later device login rather than duplicated. A
live (not stale) device key's same-machine re-login still revokes
and prints the replacement confirmation. invalid_grant/too_many_keys
is covered alongside the other terminal codes. A SIGINT with a
seeded config leaves the file byte-identical. /users/me stand-in
bodies are typed from the contract (UserMeResponse) and re-parsed
against userMeResponseSchema before being served, so a contract
drift fails the file instead of silently serving a stale shape. The
polling-gap assertion is loosened to the actual invariant (slow_down
roughly doubles the gap) rather than a fixed millisecond floor.

pollForDeviceToken/pollDeadlineMs take an optional graceSeconds,
defaulting to DEVICE_CONFIRM_GRACE_SECONDS and never passed by the
real CLI entry point. A new unit test calls it with graceSeconds: 0
against a closed port and gets "unreachable" in about a second,
covering the deadline-reached give-up branch the CLI stand-in test
cannot reach without waiting out the real multi-minute window; the
CLI test file notes why at the relevant section.
* feat: add account_kind migration and ADR 0048

* feat: exempt orcid gap by account kind, not role

* feat: refuse non-person kinds in device flow and self-mint

* feat: add admin routes for account kind and non-person keys

* feat: gate real dataset creation for test-kind accounts

* feat: add CLI support for account kinds and admin keys

* test: add live-tier probe for owner key mint

* test: pin new admin API exports in the surface test

* style: apply biome formatting to phase 4 test files

* feat: typed refusals for the kind route

POST /admin/users/:username/kind now answers { error, message } for all
four refusals (same_kind, orcid_linked, own_account,
kind_changed_concurrently) instead of an ad hoc code, so a caller reads
one shape everywhere. orcid_linked's message names `nemar auth profile
orcid unlink` directly. client.ts's prefersMessage learns the new codes
so the CLI surfaces the server's sentence, not a generic fallback.
KIND_CHANGE_GUARDED_UPDATE_SQL moves to module scope so its guarded
UPDATE can be exercised directly at the SQL level (the real-engine
substitute for a concurrent race bun:sqlite's single-writer harness
cannot reliably produce -- documented in this file's header after a
20x Promise.all experiment came back 200/200 every time).

Epic #1272 phase 4, #1284 review; ADR 0048.

* feat: CLI account-kind filter, doctor check, and key-mint hint

- nemar admin users --kind now re-filters the listing client-side, so
  a backend that still ignores ?kind= cannot mislabel it (mirrors the
  --awaiting-approval / --no-upload-access pattern).
- nemar admin doctor kinds: a new, read-only check that every
  operational account (OPERATIONAL_ACCOUNT_KINDS, exported from
  shared/contract/user.ts so the migration test, the seed script's
  test, and this check stop drifting independently) still carries its
  expected kind; migration 0082's header now notes that the SQL copy
  of this list is separate from the one TypeScript source.
- nemar admin keys create appends a retry-safety hint on a network
  failure (statusCode === 0): the key may have minted before the
  connection dropped.
- --kind example added to `admin users --help`.
- test/api-export-surface.unit.test.ts: pin the new
  getAdminUserByUsername export.

Epic #1272 phase 4, #1284 review; ADR 0048.

* fix: distinct log prefix for schema-shaped webSession errors

webSessionMiddleware's catch block now tells a schema-shaped failure
(no such column/table -- a migration that has not landed on this
environment yet) apart from a plain transient lookup failure, each
with its own console.error prefix ([web-session-mw] SCHEMA vs
[web-session-mw]), so the two stop being indistinguishable in logs.

Epic #1272 phase 4, #1284 review; ADR 0048.

* refactor: close AccountKind types; factor device-auth's liveness check

- DeviceAuthAccountRow.account_kind (and AdminKeyTargetRow through it),
  realDatasetCreateGate's input, and upload.ts's inline SELECT row all
  narrow from a bare string to the closed AccountKind type.
  identity_conflict narrows from number | boolean to number: every
  construction of DeviceAuthAccountRow is a raw D1 read, which is
  always a number.
- ProfileGapAccount.account_kind drops its | string escape hatch for
  the same reason; the one test that deliberately fed a bogus "member"
  value now casts it explicitly, with a comment explaining the cast
  defeats the type on purpose to prove the exemption still fails
  closed.
- applyServerUser's account_kind param and Config.accountKind both
  reuse AccountKind (via ContractUser / accountKindSchema) instead of
  a bare string.
- livenessStatusRefusal factors the pending/inactive-status prologue
  accountRefusal and accountLivenessRefusal both start with into one
  helper, so their shared ordering is structural rather than two
  functions kept in step by hand. The null/deleted check stays
  duplicated in each caller: it is what lets TypeScript narrow the row
  to non-null before the shared helper runs.

Epic #1272 phase 4, #1284 review; ADR 0048.

* docs: correct ADR 0048's service-account and verified-iD claims

Two factual corrections from #1284 review:

- ADR 0048 and backend/test/upload-gate.unit.test.ts both claimed a
  service account "cannot reach [the dataset-create] route at all", on
  the theory that the device/key gates stop it. False: those gates
  only stop a service/test account from SELF-serving a key
  (self-minting, or the device flow's own sign-in). A key an owner
  minted for it through POST /admin/users/:username/keys authenticates
  through authMiddleware exactly like any other -- it never reads
  account_kind -- and reaches the dataset-create route, and every
  other authenticated route, normally. The ADR now explains that a
  service account is unrestricted at realDatasetCreateGate on purpose:
  that is what an operational/automation account is for.
- The claim that yahya/arnodelorme "already hold verified iDs, so
  nothing regresses" is qualified as "at the time of writing" and
  noted as an observation about the current catalog, not something
  the migration or the kind exemption depends on.

ADR 0048.

* test: interpolate MAX_LIVE_API_KEYS; cover DELETE for a person target

- The too_many_keys test name hardcoded "25"; it now interpolates
  MAX_LIVE_API_KEYS so the name cannot drift from the real cap.
- DELETE /admin/users/:username/keys/:id had a person-kind case only
  in the GET describe block -- every DELETE case seeded a service
  target, despite the file's own docstring promising DELETE works for
  any kind. Added the missing person-kind revoke case.

#1284 review; ADR 0048.

* test: prove migration 0082's tombstone gate and case-insensitivity

Two tests that could not previously fail for the reason they claimed
to test:

- "a tombstoned row is untouched" seeded a username the migration's IN
  (...) list never named ("nemarAdmin-deleted"), so the deleted_at
  gate it claimed to cover was never exercised. Now seeds the row
  under the EXACT username the migration lists ("nemarAdmin") with
  deleted_at stamped, in its own fresh database (migration 0080's
  case-insensitive unique index on username would otherwise collide
  with the shared fixture's live row of the same name).
- "case-insensitively" never actually seeded an inverted-case
  username. Now seeds "NEMARADMIN" in its own fresh database and
  proves the COLLATE NOCASE match.

Usernames are now derived from OPERATIONAL_ACCOUNT_KINDS (shared/
contract/user.ts) instead of a hand-copied list, so this file and the
migration's own username list can no longer drift from each other
independently of the SQL.

#1284 review; ADR 0048.

* test: verify seed-dev-db.sql's account-kind statements directly

scripts/seed-dev-db.sql's account_kind UPDATEs had no test: nothing
proved the SQL text itself, as opposed to a hand-retyped copy of it,
assigns the right kind to the right usernames. ACCOUNT-KIND-SEED
marker comments let the new test slice the real statements out of the
file with readFileSync and execute them verbatim against a real
bun:sqlite database, rather than retyping them.

#1284 review; ADR 0048.

* test: exempt service/test kinds in the two untested gap readers

profile-gaps-matrix.test.ts and profile-gaps-route.test.ts proved the
kind exemption on GET /users/me, GET /auth/me, and the upload-access
refusal, but two other readers of the same account_kind column --
POST /auth/code/verify (builds its row inline) and PATCH /auth/profile
via fetchPublicUserById -- had no coverage at all. Both name
account_kind in their own hand-written SELECT, so nothing forced them
to agree with the SELECT the already-tested routes use.

#1284 review; ADR 0048.

* test: add test-kind variant of the confirm-then-collect kind change

The kind-change-between-confirm-and-collect diagnosis test only
exercised a change to 'service'. ADR 0048 has accountRefusal answer
the same service_account code for either non-person kind; this adds
the 'test' variant as proof the diagnosis path was not accidentally
written to special-case only 'service'.

#1284 review; ADR 0048.
@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR1291

Version: 0.9.17-PR1291.3bf7aef
Commit: 3bf7aef


🤖 Updated on each push.

@neuromechanist
neuromechanist merged commit 0085d0a into dev Sep 8, 2026
23 checks passed
@neuromechanist
neuromechanist deleted the feature/issue-1272-epic-orcid-cli-signin branch September 8, 2026 14:19

This branch was previously deployed

1 inactive deployment
npm — 3bf7aef2 Deployed Sep 8, 2026 by neuromechanist via publish #2361
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant